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EXECUTIVE  SUMMARY 


An  effort  was  conducted  to  determine  actual  ground-to-air,  and  air-to-ground  performance  of  the 
Airline  Communications  and  Reporting  System  (ACARS),  Very  High  Frequency  (VHF)  Data 
Link  system.  Parameters  of  system  throughput,  error  rates,  and  availability  were  measured  by 
tabulating  statistics  of  messages  ranging  from  2  to  150  bytes  in  length.  The  intervals  of 
transmission  were  developed  based  on  anticipated  air  traffic  service  (ATS)  requirements  for 
tactical  air  traffic  control  (ATC)  messages  and  their  associated  replies. 

Overall,  the  average  round  trip  message  delay  fell  in  the  range  of  10  to  20  seconds,  with  5  out  of 
approximately  2300  messages  lost. 

Aeronautical  Radio  Inc.  (ARINC)  did  not  endorse  these  tests,  indicating  that  the  Federal  Aviation 
Administration  (FAA)  tests  did  not  take  advantage  of  the  capabilities  of  the  ACARS  network 
which  has  been  optimized  for  airline  use. 


1. 


OBJECTIVE  OF  THE  TESTS. 


1.1  DESCRIPTION. 

Demonstrate  that  the  Airlike  Communications  and  Reporting  System  (ACARS)  Very  High 
Frequency  (VHF)  Data  Link  is  robust  enough  to  satisfy  Air  Traffic  System  (ATS)  applications 
performance  requirements.  Through  flight  tests  with  ACARS  avionics  and  the  Aeronautical 
Radio  Incorporated  (ARINC)  system,  specific  parameters  that  characterize  the  system  quality  of 
service  (QOS)  were  experimentally  examined. 

1 .2  DEFINITIONS  OF  QOS  PARAMETERS . 

Parameters  of  QOS  applicable  to  this  evaluation  program  are  defined  in  this  section. 

1.2.1  Transit  Delay. 

This  is  a  measure  of  the  time  interval  beginning  with  message  composition  at  the  source  machine, 
and  ending  with  the  receipt  and  recognition  of  the  message  at  the  destination  machine. 

In  this  test,  transit  delay  was  measured  separately  for  the  uplink  and  downlink  paths. 

Based  on  data  reported  by  ARINC,  expected  transit  times  are: 

Average  Transit  Delay  for  Downlinks:  9  seconds 

Average  Transit  Delay  for  Uplinks:  12  seconds  * 

*  Note:  3  seconds  attributed  to  polling  the  ground  stations. 

Transit  Delay  is  affected  by: 

a.  Type  of  the  message: 

i.  length 

ii.  Type 

b.  Type  of  radio 

c.  Channel  utilization 

1 .2.2  Residual  Error  Rate. 

This  parameter  is  a  measure  of  lost  or  garbled  messages,  expressed  as  a  fraction  of  the  total  sent 
in  the  sampling  period. 

ARINC  document  D001 10,  “Message  Integrity  Across  the  ACARS  Network,”  defines  the 
domains  in  which  undetected  errors  occur  and  the  probability  of  occurrence. 

Based  on  data  reported  by  ARINC,  end-to-end  integrity  with  error  correction  is: 


1 


-  Parity  Check  for  Character 

-  Frame  Level  Cyclic  Redundancy  Check  (CRC) 

Pe  =  10 

-  Transport  Layer  CRC 

Pe=  10 

-  Properly  Selected  Transport  Layer  CRC 

Pe=  10‘ 

End-to-end  data  integrity  is  affected  by: 

a.  Type  of  the  message: 

i.  message  length 

ii.  Type 

b.  Error  detection  and  protection  codes  used 

2.  TEST  ENVIRONMENT. 

2.1  TESTS  CONDUCTED. 

2.1.1  QOS  Evaluation. 

Send  M  series  of  N  messages  each  T  seconds  and  compute  the  transit  delay  of  each  message. 
Series  will  differ  in  length  of  the  message,  and  include  message  lengths  of  55, 110,  and  220 
characters.  Each  message  was  time  tagged  according  to  the  format  DD:HH:MM:SS,  when  the 
message  was  transmitted  by  the  communications  management  unit  (CMU).  Time  tagging  was 
provided  by  a  function  within  the  management  unit  (MU),  and  is  intended  to  mark  the  time  of 
transmission  of  the  message. 

The  number  of  messages  sent  within  a  series  varied  from  180  messages  to  1,020  messages. 

2.1.2.  Residual  Error  Rate. 

Examine  the  message  set  described  in  section  2.1.1.  above  in  post-test  analysis,  to  determine  by 
comparison  of  log  files,  the  ratio  of  unsuccessful  transmissions. 

For  each  message  sent  and  received,  CRC  was  calculated  and  compared  to  the  message  contents. 
2.2  INSTALLED  SOFTWARE  CAPABILITIES. 

Software  functionality  was  developed  for  the  ACARS  tests  which  were  conducted  in  March  1992. 
The  software  has  the  following  capabilities: 

2.2. 1  Time  Stamp. 

Every  message,  sent  or  received  by  the  ground  equipment  has  its  own  valid  time  stamp.  Time 
stamps  generated  by  the  ground  equipment  were  compared  to  the  MU  generated  time  stamps  in 
the  analysis  of  transit  delay. 
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2.2.2  Logout  Files. 


Every  event  within  the  ground  equipment  (either  outgoing  or  incoming),  including  messages, 
acknowledgments  received,  control  information,  etc.,  was  automatically  stored  on  disk  files  when 
the  event  was  displayed  on  the  ground  station  control  console. 

2.2.3  Free  Text. 

A  capability  was  provided  to  enable  the  transmission  of  free  text  messages  up  to  220  characters  in 
length,  containing  data,  control  information,  or  any  other  text.  An  example  format  is  shown  in 
figure  1. 


Format:  send  <  destination  station  >  <  user  text  > 

Example:  send  n39  ***  FAA  TECH  CENTER  DEC  TEST  *** 

which  sends  to  the  line: 

QU  DDLXCXA 
.ACYXGXA  062311 
CMD 

/AN  N39  /AP  ACY 

-  QUACYXGXA  ~FAA  TCH  ***  FAA  TECH  CENTER  DEC  TEST  ***  BBOA 
where  BBOA  is  the  computed  CRC  of  the  message. _ 


FIGURE  1.  EXAMPLE  MESSAGE  FREE  TEXT 


2.2.4  Loopback  Messages. 

When  software  in  the  MU  detected  the  sequence  of  characters  “tilde4”  (~4)  in  a  specific  portion 
of  the  message,  it  automatically  sends  the  message  back  to  the  originator. 

The  ground  station  at  the  Technical  Center  also  has  the  capability  to  send  loopback  messages  to 
the  MU  installed  on  the  aircraft  and  store  the  replies  for  analysis.  An  example  format  is  shown  in 
figure  2. 
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Format: 


loopback  <  station  >  <times>  @  <interval>  <user  text> 
Example:  loopback  n39  3@5  ***  FAA  TECH  CENTER  DEC  TEXT  *** 


which  will  send  to  the  line: 

QU  DDLXCXA 
.ACYXGXA  062313 
CMD 

/ANN39/APACY 

-  QUACYXGXA  ~4[L00PBACK  004:00000:0699923635] 
***  FAA  TECH  CENTER  DEC  TEST  ***  FFA6 


wait  5  seconds  and  send: 


QU  DDLXCXA 
.ACYXGXA  062314 
CMD 

/AN  N39  /AP  ACY 

-  QUACYXGXA  ~4[LOOPBACK  004:00001:0699923640] 
***  FAA  TECH  CENTER  DEC  TEST  ***  72E1 


wait  5  seconds  and  send: 

QU  DDLXCXA 
.ACYXGXA  062314 
CMD 

/AN  N39  /AP  ACY 

-  QUACYXGXA  ~4[L00PBACK  004:00002:0699923645] 
***  FAA  TECH  CENTER  DEC  TEST  ***  1FOB 


FIGURE  2.  EXAMPLE  MESSAGE  LOOPBACK  TEST 

The  same  sequence,  tilde4,  was  also  recognized  by  the  ground  equipment  to  cause  a  loopback  to 
the  avionics.  All  messages  were  time  tagged  and  appended  with  a  CRC  sequence  just  prior  to 
being  looped  back. 

To  prevent  the  ground  station  from  looping  back  messages  if  not  desired,  the  following  command 
is  available:  set  loopback  ~4  off 
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2.2.5  Error  Control. 


To  detect  messages,  undetected  by  ACARS,  or  to  detect  errors  in  the  case  where  the  message  is 
corrupted  in  transmission,  the  ground  station  had  the  capability  to  compute  and  append  CRC 
codes  to  each  message. 

The  following  commands  enable  these  capabilities: 
set  arinc  on 

which  applies  the  polynomial:  xl6+xl5+xl3+x5+x3+x'+l 

set  International  Telephone  and  Telegraph  Consultative  Committee  (ccitt)  on 
which  applies  the  polynomial :  x 1 6+x 1 2+x5+ 1 

Note:  The  probability  of  an  undetected  error  is  greater  if  the  polynomial  applied  at  the  transport 
layer  is  the  same  used  at  the  link  layer  (CCITT). 

The  error  control,  if  enabled  computes  the  CRCs  of  every  message  to  be  sent,  regardless  of  the 
type  of  message,  regular  or  loopback  type. 

2.2.6  Flow  Control. 

In  order  to  prevent  excessive  queuing  in  the  ARINC  network  due  to  message  overloading  (see 
appendix  A),  flow  control  software  capabilities  have  been  included  in  both  air  and  ground  end 
systems.  A  sliding  window  protocol  was  used  for  flow  control,  that  prevents  more  than  five 
messages  from  queuing  in  the  previous  loopback  message  that  have  not  been  echoed  by  the  peer 
equipment.  In  case  of  overload,  and  if  no  echo  is  received,  transmission  of  new  messages  is 
postponed  until  a  message  is  received  or  the  corresponding  validity  timer  of  the  message  expires. 

2.3  HARDWARE  DESCRIPTION. 


a.  Ground  Terminal 

-  Digital  DEC  Station  2100,  Digital  Ultrix  Operating  System 

-  Air  Land  Systems  SA-300  Air  Land  Equipment 

-  Racal-Milgo  122-RALA  Modem 

b.  Avionics 

-  Teledyne  Controls  Management  Unit  (MU),  ARINC 
Characteristic  724 

-  Interactive  Display  Unit  (IDU) 

-  Collins  VHF  Radio,  ARINC  Characteristic  716 
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-  Long  Ranger  FP/PLUS  Loran-C  Receiver  (not  used) 

-  Ziatech  STD-80  Bus  Personal  Computer,  STD-DOS  Operating 
System  Emulates  Digital  Flight  Data  Acquisition  Unit  (DFDAU) 

3.  FLIGHT  TEST  SCHEDULES. 

3.1  ITINERARY  AND  DURATION  OF  TESTS. 

See  figure  3  for  definition  of  NYC-DCA-CHI  corridor. 

March  9:  Flight  to  Boston,  Albany,  Atlantic  City 

Duration:  3  hours,  14  minutes 

March  10:  Flight  to  Norfolk,  Charleston,  Atlantic  City 

Duration:  3  hours,  14  minutes 

March  1 1 :  Flight  to  Cincinnati,  Dayton,  Cleveland,  Atlantic  City 

Duration:  5  hours  1  minute 

March  12:  Flight  to  Chicago,  Atlantic  City 

Duration:  4  hours,  4  minutes 

3.2  TESTS  PERFORMED. 

March  9:  300  55-character  messages  (3  series) 

200  1 10-character  messages  (2  series) 

March  10:  300  55-character  messages  (3  series) 

180  110-character  messages  (2  series) 

March  11:  340  55-character  messages  (4  series) 

300  1 10-character  messages  (3  series) 

March  12:  398  55-character  messages  (4  series) 

300  1 10-character  messages  (3  series) 

Totals:  1298  55-character  messages 

1020  1 10-character  messages 
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nf^A 


NYC-DCA-CHI 

CORRIDOR 


NOTE:  LOCATIONS  LISTED 
DESIGNATE  CRITICAL  AIRPORTS 


FIGURE  3.  ACARS  -  DEFINITION  OF  NYC-DCA-CHI  CORRIDOR 
4.  RESULTS. 

4. 1  QUALITATIVE  COMMENTED  RESULTS . 

The  analytical  information  computed  for  each  series  of  loopback  messages  has  the  following 
format: 

Month:  Day: 

Series:  n/Total 

Type:  N  messages  of  k  characters 
Statistics: 

Mean:  average  of  the  transit  delays  computed  from  the  sample 
Mode:  most  frequent  value  observed 
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St  Dev: 


standard  deviation  of  the  transit  delays  computed  from  the  sample 


Minimum:  best  transit  delay  observed 

Maximum:  worst  transit  delay  observed 

Problems  observed: 

Lost:  lost  messages,  if  any 

Recovery:  parameter  that  measures  the  capability  of  the  system  to  restore 

normal  operation  after  an  overloading  scenario. 

This  parameter  is  computed  as  the  time  in  seconds  elapsed  since  the  maximum  delay  is  reached 
until  the  delay  decreases  to  a  value  equal  or  smaller  than  the  mean  of  the  sample. 

March  9  Series:  1/5 

Type:  100  messages  of  55  characters 

Statistics: 

Mean:  12.96  secs 

Mode:  7  secs  (35  occurrences) 

St  Dev:  9.21  secs 

Minimum:  7  secs  (35  occurrences) 

Maximum:  52  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  52  to  8:  60  secs 

March  9  Series:  2/5 

Type:  100  messages  of  1 10  characters 

Statistics: 

Mean:  21.68  secs 

Mode:  12  secs  ( 1 8  occurrences) 

St  Dev:  20.02  secs 

Minimum:  7  secs  (14  occurrences) 

Maximum:  95  secs  ( 1  occurrence) 

Problems  observed: 

Lost:  1  messages 

Recovery:  From  95  to  19:  220  secs 


March  9 


Series:  3/5 


March  9 

Type: 


March  9 


Type:  100  messages  of  55  characters 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 

Problems  observed: 
Lost: 
Recovery: 

Series:  4/5 


12.28  secs 

7  secs  (33  occurrences) 
7.12  secs 

6  secs  (3  occurrences) 
37  secs  (3  occurrences) 


none 

From  37  to  9:40  secs 


100  messages  of  1 10  characters 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 


36.49  secs 

12  secs  (26  occurrences) 
42  04  secs 

7  secs  (4  occurrences) 
172  secs  (1  occurrence) 


Problems  observed: 

Lost:  none 

Recovery:  From  172  to  34:  340  secs 


Series:  5/5 


Type:  100  messages  of  55  characters 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 


14.25  secs 

12  secs  (32  occurrences) 
11.05  secs 

6  secs  (5  occurrences) 

76  secs  (1  occurrence) 


Problems  observed: 

Lost:  none 

Recovery:  From  76  to  13:  100  secs 
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March  10  Series:  1/5 


March  10 


March  10 


Type:  100  messages  of  55  characters 
Statistics: 

Mean:  32.92  secs 

Mode:  7  secs  (22  occurrences) 

St  Dev:  34.03  secs 

Minimum:  7  secs  (22  occurrences) 

Maximum:  122  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  122  to  45:  180  secs 

Series:  2/5 

Type:  100  messages  of  1 10  characters 
Statistics: 

Mean:  17.02  secs 

Mode:  7  secs  (29  occurrences) 

St  Dev:  14.54  secs 

Minimum:  7  secs  (29  occurrences) 

Maximum:  82  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  82  to  30:  120  secs 

Series:  3/5 

Type:  100  messages  of  55  characters 
Statistics: 

Mean:  18.20  secs 

Mode:  7  secs  (29  occurrence) 

St  Dev:  13.06  secs 

Minimum:  6  secs  (1  occurrence) 

Maximum:  62  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  62  to  18:  160  secs 


10 


March  10 


Series:  4/5 


March  10 


March  1 1 


Type:  100  messages  of  55  characters 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 


130.85  secs 

6  secs, 7  secs,  171  secs  (4  occurrences) 
94.56  secs 

6  secs  (4  occurrences) 

372  secs  (1  occurrence) 


Problems  observed: 

Lost:  4  messages 

Recovery:  From  372  to  128:  420  secs 


Series:  5/5 


Type:  80  messages  of  1 10  characters 

Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 

Problems  observed: 

Lost:  none 

Recovery:  From  32  to  13:  60  secs 

Series:  1/7 

Type:  100  messages  of  55  characters 
Statistics: 

Mean:  68.95  secs 

Mode:  12  secs  (8  occurrences) 

St  Dev:  48.60  secs 

Minimum:  7  secs  (4  occurrences) 

Maximum:  217  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  217  to  63:  300  secs 


14.60  secs 

12  secs  (35  occurrences) 
4.42  secs 

9  secs  ( 1  occurrence) 

32  secs  (1  occurrence) 
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March  1 1 


Series:  2/7 


March  1 1 


March  11 


Type:  100  messages  of  55  characters 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 

Problems  observed: 
Lost: 
Recovery: 

Series:  3/7 

Type:  100  messages 

Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 

Problems  observed: 
Lost: 
Recovery: 

Series:  4/7 

Type:  100  messages 

Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 

Problems  observed: 
Lost: 
Recovery: 


29.23  secs 

12  secs  (12  occurrences) 
16.92  secs 

7  secs  (S  occurrences) 

77  secs  (2  occurrences) 


none 

From  77  to  27:  100  secs 


of  1 10  characters 


33.58  secs 

12  secs  (17  occurrences) 
25.68  secs 

7  secs  (7  occurrences) 
118  secs  (1  occurrence) 


none 

From  1 18  to  23:  180  secs 


of  1 10  characters 


18.92  secs 

12  secs  (28  occurrences) 
10.66  secs 

7  secs  (4  occurrences) 
55  secs  (1  occurrence) 


none 

From  55  to  13:  80  secs 
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March  1 1 


Series:  5/7 


March  1 1 


March  1 1 


Type:  100  messages  of  110  characters 
Statistics: 

Mean:  34.92  secs 

Mode:  7  secs  (22  occurrences) 

St  Dev:  37.74  secs 

Minimum:  7  secs  (22  occurrences) 

Maximum:  162  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  162  to  58:  240  secs 

Series:  6/7 

Type:  100  messages  of  5S  characters 

Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 

Problems  observed: 

Lost:  none 

Recovery:  From  87  to  12:  160  secs 

Series:  7/7 


21.96  secs 

12  secs  (14  occurrences) 
16.99  secs 

6  secs  (7  occurrences) 

87  secs  (1  occurrence) 


Type:  40  messages  of  55  characters 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 


16.08  secs 

12  secs  (27  occurrences) 
9.65  secs 

12  secs  (27  occurrences) 
57  secs  (1  occurrence) 


Problems  observed: 

Lost:  none 

Recovery:  From  57  to  13:  80  secs 


13 


March  12 


Series:  1/7 


Type:  100  messages  of  55  characters 
Statistics: 

Mean:  19.81  secs 

Mode:  12  secs  (29  occurrences) 

St  Dev:  10.98  secs 

Minimum:  7  secs  (6  occurrences) 

Maximum:  52  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  52  to  13:  60  secs 

March  12  Series:  2/7 

Type:  100  messages  of  55  characters 

Statistics: 

Mean:  13.77  secs 

Mode:  7  secs  (30  occurrences) 

St  Dev:  9.15  secs 

Minimum:  6  secs  (1 1  occurrences) 

Maximum:  52  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  52  to  12:  60  secs 

March  12  Series:  3/7 

Type:  100  messages  of  55  characters 

Statistics: 

Mean:  11.58  secs 

Mode:  7  secs  (54  occurrences) 

St  Dev:  8.80  secs 

Minimum:  6  secs  (7  occurrences) 

Maximum:  57  secs  ( 1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  57  to  8:84  secs 
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March  12 


Series:  4/7 


Type:  100  messages  of  1 10  characters 

16.57  secs 

12  secs  (21  occurrences) 
9.74  secs 

7  secs  (15  occurrences) 
58  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  58  to  13:80  secs 

March  12  Series:  5/7 

Type:  100  messages  of  1 10  characters 

Statistics: 

Mean:  20.92  secs 

Mode:  7  secs  (25  occurrences) 

St  Dev:  24.97  secs 

Minimum:  7  secs  (25  occurrences) 

Maximum:  127  secs  (1  occurrence) 

Problems  observed: 

Lost:  none 

Recovery:  From  127  to  19:180  secs 

March  12  Series:  6/7 

Type:  100  messages  of  1 10  characters 

Statistics: 

Mean:  34.32  secs 

Mode:  7  secs  (23  occurrences) 

St  Dev:  42.42  secs 

Minimum:  7  secs  (23  occurrences) 

Maximum:  162  secs  (2  occurrences) 

Problems  observed: 

Lost:  none 

Recovery:  From  162  to  23:  220  secs 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 
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March  12 


Series:  7/7 


Totals: 


Type:  98  messages  of  55  characters 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 


14.30  secs 

12  secs  (27  occurrences) 
9.67  secs 

6  secs  (4  occurrences) 

56  secs  (1  occurrence) 


Problems  observed: 

Lost:  none 

Recovery:  From  56  to  12:  120  secs 


Type:  23 1 8  messages  of  55/1 10  characters 


Statistics: 

Mean: 

Mode: 

St  Dev: 

Minimum: 

Maximum: 


28.01  secs 

12  secs  (450  occurrences  =  19.45  %) 
38.79  secs 

6  secs  (42  occurrences) 

372  secs  (1  occurrence) 


Problems  observed: 

Lost:  5  messages 

Recovery:  Best:  From  37  to  9:  40  secs 

Worst:  From  372  to  128:  420  secs 


4.2  DISCI  JSSION  OF  RESULTS. 

Figure  4  shows  the  combined  total  of  messages  transmitted  between  aircraft  and  ground  station 
peer  entities,  over  the  4-day  period. 
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FIGURE  4.  TESTS  PERFORMED 


Figure  5  shows  the  bin  distribution  of  one-way  delay  times  (in  seconds),  by  day,  over  the  4-day 
period.  All  data  samples  are  combined  into  figure  5.  Most  of  the  message  transit  times  fall  into 
the  1 1-  to  20-second  bin  range,  with  the  second  highest  incidence  in  the  6-  to  10-second  range. 


FIGURE  5.  DISTRIBUTION  OF  FREQUENCY  OF  OCCURRENCE  OF 
ONE-WAY  DELAY  TIMES 
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Figure  6  shows  in  a  different  presentation,  accumulated  delay  data  by  day,  for  the  4-day  period. 


FIGURE  6.  AVERAGE  MESSAGE  DELAY 


Figure  7  shows  one-way  delay  times,  in  seconds,  averaged  over  each  series.  The  coordinate 
references  on  the  horizontal  axis  refer  to  the  day  and  series  number  within  the  day.  For  example, 
s9.1  refers  to  the  first  message  series  on  March  9,  1992.  Average  series  delays  in  excess  of  120 
seconds  occurred  on  March  10,  1992,  during  a  flight  to  Norfolk,  Virginia.  These  extended  transit 
times  were  the  result  of  queuing,  even  with  flow  control  procedures  in  place. 


Figure  8  shows  the  standard  deviation  of  transit  times  for  each  series,  expressed  in  the  same 
format  as  figure  7.  The  shape  factor  of  the  curve  resembles  figure  7. 


FIGURE  8.  STANDARD  SERIES  DEVIATION 


5.  SUMMARY  OF  RESULTS. 

One-way  transit  times  in  the  range  of  6  to  20  seconds  formed  the  preponderance  of  data.  Longer 
delay  times  were  the  result  of  queuing  within  the  ACARS  network.  Queuing  occurred  when  a 
message  was  delivered  to  the  network  for  transmission  before  the  previous  message  was  closed 
out. 

ARINC  reports  that  message  queuing  represents  misuse  of  the  network,  and  represents  an 
enormously  high  amount  of  traffic.  ARINC  correspondence  on  the  subject  is  contained  in 
appendix  A. 

Short  message  lengths  (55  characters)  chosen  by  the  FAA  Technical  Center  test  personnel  were 
believed  to  represent  typical  ATS  message  lengths  after  necessary  protocol  and  routing 
information  were  added.  Longer  message  lengths  (110  bytes)  were  believed  to  represent  longer 
messages  of  flight  information  or  weather. 

A  total  of  5  messages  were  lost  out  of  a  total  of  2,318  messages  transmitted.  Of  these,  1  message 
was  1 10  characters  in  length,  and  4  were  55  characters  in  length.  All  4  of  the  lost  55  character 
messages  were  lost  during  the  same  flight  which  occurred  on  March  10, 1992. 

6.  CONCLUSIONS. 

Data  contained  in  this  report  provides  a  performance  indicator  of  the  performance  of  the  Airline 
Communications  and  Reporting  System  (ACARS)  network.  Although  commercial  avionics  were 
used  in  this  test,  Aeronautical  Radio  Inc.  (ARINC)  does  not  endorse  the  results,  stating  that  the 
network  was  not  used  in  a  manner  for  which  it  is  designed. 


APPENDIX  A 

ARINC  PHASE  4  TEST  REPORT 


RFAA/TELEDYNE  724  ACARS  AVIONICS 
AQP  PHASE  4  FLIGHT  TEST 
23  MARCH  1992 


GENERAL: 

ARINC  conducted  a  Phase  4  test  audit  for  the  VHF  ACARS  avionics  aboard  an  FAA/CV-580 
aircraft  (reg.  ID.  3)  on  11  March  1992.  The  aircraft  was  being  used  for  Intensive  end-to-end  Data 
Link  message  integrity  testing  by  the  FAA  Technical  Center  in  Atlantic  City,  NJ.  Rick  Olson  of 
the  FAA  served  as  the  on-board  engineer.  The  aircraft  was  operated  locally  in  the  NJ-NY-MASS 
area.  Limited  testing  of  supported  message  labels  was  performed  due  to  the  enormous  amount  of 
Loop-Back  testing  being  performed. 

SUMMARY: 

Phase  4  flight  testing  was  conducted  on  March  11, 1992,  for  the  FAA  Teledyne  avionics  installed 
aboard  the  FAA’s  CV-580/N39  aircraft.  Only  limited  testing  of  the  avionics’  supported  message 
labels  could  be  performed  due  to  the  intensive  FAA  Loop-Back  testing  during  the  audit.  In  spite 
of  the  poor  signal  strengths  received  during  the  course  of  the  audit,  the  avionics  uplink  success 
ratios  were  sufficiently  above  the  88%-minimum  AQP  requirement.  However,  a  significant  error 
occurred  with  the  ACK/NAK  protocol  exhibited  by  the  avionics.  In  each  case,  when  an  RAT  1 
(Uplink  Display  Message)  was  initiated  for  the  aircraft,  the  avionics  improperly  responded  by 
providing  an  acknowledgment  followed  by  a  nonacknowledgment  for  the  same  message.  It  is 
unknown  if  the  avionics  received  any  or  all  of  these  display  messages.  In  light  of  this  problem, 
further  testing  in  support  of  the  uplink  Labels  not  tested  during  this  audit  is  recommended  to  test 
the  remaining  system  responses. 

An  item  of  interest  is  discussed  in  the  Problems  and  Issues  Section  below  concerning  the 
reliability  of  the  rapid  Loop-Back-Integrity  testing  conducted  by  the  FAA  during  the  flight. 

MESSAGES  OBSERVED: 

The  following  uplink  and  downlink  messages  were  observed  during  the  audited  flights: 


Label 

Sublabel 

U/D 

_DEL 

n/a 

U 

General  Acknowledgment 

•y 

n/a 

U 

Autotune  to  New  Frequency 

RA 

~  1 

u 

Display  Message 

RA 

~  2 

u 

0001  Dump 

RA 

~  3 

u 

Memory  Dump 

RA 

~  4 

u 

Loop-Back  Test 

51 

n/a 

u 

GMT  Update 

54 

nidi 

D 

Voice  Go-ahead 

A-l 


Label 

Sublabel 

U/D  Message  Tvpe 

_DEL 

n/a 

D 

General  Acknowledgment 

NAK 

n/a 

D 

Non-Acknowledgment 

HI 

DF 

D 

DFDAU  Message 

QO 

n/a 

D 

Link  Test 

Q5 

n/a 

D 

Unable  to  Deliver  Message 

RB 

~2 

D 

0001  Dump 

RB 

-3 

D 

Memory  Dump 

RB 

~4 

D 

Loop-Back  Test  Response 

52 

TXT 

D 

Free  Text  Message 

51 

n/a 

D 

GMT  Update  Request 

AVIONICS  UPLINK  PERFORMANCE: 

The  following  uplink  success  ratios  were  observed  during  the  audit.  These  ratios  are  a  measure  of 
the  reliability  of  an  Individual  message  being  received  by  the  avionics  on  the  first  uplink 
transmission  occurrence.  The  observed  ratios  were  as  follows: 

Unsolicited  Uplink  Successes:  82%  (135  Successes/163  Attempts) 

Solicited  Uplink  Successes:  91%  (153  Successes/167  Attempts) 

Overall  Uplink  Successes:  87%  (288  Successes/330  Attempts) 

With  the  exception  of  an  autotune  message  (Label :;)  sent  from  ARINC  and  a  DFDAU  message 
discussed  in  the  Problems  and  Issues  section  below,  all  messages  were  eventually  acknowledged 
by  the  avionics. 

NOTED  PROBLEMS  AND  ISSUES : 

A  significant  problem  was  observed  during  this  audit  with  the  ACK/NAK  protocol.  Over  the 
course  of  the  2-hour/25-minute  audit,  every  RA/-1  (Free  Text  Uplink)  that  was  Initiated  (13 
total)  was  originally  ACK’d  by  the  avionics  and  then  NAK’d  5  seconds  later.  This  is  indicative  of 
a  significant  protocol  error  and  is  a  deviation  from  AEEC  specifications.  Once  AFEPS  received 
the  ACK  for  the  uplinked  message,  it  considered  the  message  sequence  complete  and  freed  the 
buffer  for  the  next  uplink  message.  The  messages  were  effectively  lost  if  not  properly  buffered  by 
the  avionics.  It  is  unknown  if  the  avionics  actually  processed  and  displayed  the  13  NAK’d 
uplinks. 

Received  downlink  signal  strengths  over  the  course  of  the  audit  were  generally  weak  which  may 
have  contributed  to  the  uplink  success  ratios. 
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Two  RA/-4  (Loop-Back)  messages  were  NAK’d  during  the  audit,  presumably  due  to  weak  signal 
strengths  which  were  observed  before  and  after  the  NAK  occurrences. 

On  one  occasion,  an  Hl/DF  (DFDAU)  uplink  was  Q5’d  and  returned  to  its  origin  due  to  the 
improper  usage  of  the  uplink  SMI  and  internal  sub-label  combination  (DFD/-1)  in  the  format  of 
the  input  message. 

An  Issue  of  Note  for  the  FAA:  during  the  early  portion  of  the  audit,  while  attempting  to  conduct 
Phase  4  coordination,  a  Memory  Dump  and  an  0001  Dump  uplink  were  both  initiated  by  ARINC 
while  the  avionics  was  in  the  process  of  responding  to  the  first  series  of  Loop-Back  testing 
between  the  avionics  and  the  FAA  Host  Processor.  The  avionics  was  responding  to  the  50  Loop- 
Backs  in  the  command  series  at  30-second  intervals.  The  avionics  handled  the  memory  Dump 
correctly  but  it  appears  that  it  could  not  respond  quickly  enough  to  accommodate  the  extra 
demand  of  the  0001  Dump  upon  its  arrival  between  Loop-Back  responses.  A  series  of  36  failed 
uplink  attempts  over  a  period  of  6  minutes  was  initiated  during  the  time  that  the  avionics  was  in 
the  process  of  responding  to  the  0001  and  Loop-Back  testing  simultaneously.  All  uplinks  were 
eventually  acknowledged  and  responded  to  by  the  avionics  during  this  period. 

Noting  the  overloaded  condition  of  the  avionics,  ARINC  decided  to  assume  a  passive  position 
and  allow  the  FAA  to  complete  its  Loop-Back  testing  unabated  while  continuing  to  collect  the 
audit  data  which  is  necessary  for  ARINC  Avionics  Qualification. 

Because  of  the  nonstandard  circumstances  of  the  continuous  Loop-Back  testing  and  the 
acceptable  performance  of  the  avionics  responding  to  subsequent  Loop-Back  testing  at  even 
tighter  intervals  (100  messages  at  20-second  intervals),  these  failures  were  not  included  in  the 
success  ratio  calculations  above:  however,  it  is  recommended  that  for  qualification  purposes 
further  testing  be  performed  to  include  the  message  labels  which  were  not  tested  during  this  flight 
test. 

Since  20-second  intervals  between  messages  delivered  to  a  single  aircraft  is  not  representative  of  a 
typical  scenario,  data  results  from  the  FAA’s  perspective  may  be  affected.  The  normal  handshake 
and  timeout  retransmission  intervals  of  the  system  could  increase  the  likelihood  of  message  delays 
and  undelivered  uplinks. 

TEST  LIMITATIONS: 

Because  of  the  enormous  amount  of  Loop-Back  messages  being  exchanged  during  the  audit,  all 
message  label  types  could  not  be  tested. 
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ARIIMO 

A£PC**ajtcal  Radio  Inc 


2551  Riva  Road 

Annapolis,  Maryland  21401-7465 


April  23,  1992 
File:  07-1-7  FA 

Mr.  Rick  Olsen 
Avionics  Engineer 
FAA  Technical  Center 
Airborne  Systems  Technology 
Branch,  ACD-330 
Atlantic  City  Intn’l.  Airport 
Atlantic  City,  NJ  08405 

Dear  Rick: 

As  a  follow-up  to  our  recent  phone  conversation,  I  would  like  to  reiterate  the  following 
points. 

The  Avionics  Qualification  Policy  (AQP)  tests  on  your  VHF  avionics  have  not 
been  completed. 

Use  of  the  ARINC  operational  network  for  on-line  testing  is  not  permitted. 

On  March  9,  1992  we  attempted  a  phase  4  test  of  your  avionics.  This  test  was 
incomplete,  partially  because  of  the  hundreds  of  loop-back  tests  that  you  sent  within  an 
unusually  short  time-frame.  We  consider  any  test  results  that  you  obtain  with  this 
avionics  to  be  invalid  until  the  AQP  program  is  completed. 

We  are  anxious  to  assist  you  in  your  evaluation  of  data  link  and  would  appreciate  your 
^7  timely  scheduling  of  phase  4  testing. 

You  should  understand  that  we  have  optimized  the  ACARS  system  for  standard  flight 
profiles.  For  instance,  the  downlink  buffer  size  has  been  tailored  for  the  number  and 
rate  of  arrival  of  downlinks  that  we  have  learned  to  expect  from  an  operational  airline 
flight.  When  these  thresholds  are  exceeded,  alarms  to  our  System  Manager  are 
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Mr.  Rick  Olsen 
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generated  and  if  the  condition  is  not  corrected,  system  performance  is  impacted.  This 
was  the  case  when  you  transmitted  hundreds  of  loop-back  messages  on  March  9th.  Any 
non-standard  use  of  ACARS  must  be  coordinated  with  the  ARINC  System  Management 
Office.  Any  ACARS  use  that  resembles  a  stress  test  will  not  be  permitted  without 
thoroughly  evaluating  the  impact  on  critical  airline  operational  traffic. 

Veiy  truly  yours, 


A-5 


APHSIC 

aeponajtic*.  Roc  !*.: 


2551  Riva  Road 

Annapolis,  Maryland  21401-7465 


June  17,  1992 
File:  07-1-7  FA 


Mr  Rick  Olson 

Avionics  Engineer 

FAA  TECHNICAL  CENTER 

Airborne  Systems  Technology  Branch,  ACD-330 

Atlantic  City  International  Airport,  NJ  08405 


Dear  Rick: 


John  J.  Sullivan 

Vice  President 
Quality  Management 


Attached  is  the  completed  Phase  4  AQP  test  report  for  the  FAA/Teledyne  724 
ACARS  avionics.  As  noted  in  the  report,  the  avionics  has  a  significant  error  with 
ACK/NAK  response  protocol  logic.  The  problem  was  evident  when  responding  to 
the  RA/~  1  and  RA/-4  uplink  labels.  Each  RA/~  1  message  improperly  received 
both  an  ACK  and  a  NAK  for  the  first  block  of  each  uplink  message,  while  each 
RA/~4  message  received  an  ACK  for  the  first  block  and  both  an  ACK  and  a  NAK 
for  any  additional  blocks. 


Since  multiple  responses  to  a  single  uplink  are  not  in  accordance  with  any  of  the 
AEEC  specifications  and  because  they  cause  system  inefficiencies  due  to  the 
superfluous  downlinks  and  unnecessary  uplink  retransmissions  that  occur,  we  can 
not  approve  this  release  for  operation  on  the  ACARS  network.  Please  contact  us 
with  scheduling  information  regarding  AQP  testing  on  the  next  release  for  this 
software. 


We  appreciate  you  cooperation  with  the  AQP  process  and  believe  that  your 
participation  is  helping  to  maintain  and  augment  the  high  degree  of  network 
reliability  for  all  ACARS  users. 


Sincerely, 


jg1 

cc:  Angus  McEachen 

92-24.FA 
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FAA/TELEDYNE  724  ACARS  AVI0NIC8 
AQP  PHASE  4  FLIGHT  TE8T 
8RN#  92-00870 
7  KAY  1992 


GENERALS 

Coordinated  Phase  4  testing  was  conducted  on  30  April  1992  for  the 
FAA/Teledyne  724  ACARS  avionics.  This  test  was  initiated  to  test 
system  responses  which  could  not  be  verified  during  previous  flight 
testing  due  to  the  intensive  Loop-Back  testing  being  performed  by 
the  FAA.  The  test  was  ground  based  and  fully  coordinated  with  test 
engineers  at  both  ARINC  (Steve  Leger)  and  the  FAATC  (Rick  Olson)  in 
Atlantic  City  providing  the  various  supported  uplink  and  downlink 
message  labels. 


SUMMARY: 

The  FAA/Teledyne  724  avionics  performed  well  during  the  test  with 
the  serious  exception  of  the  ACK/NAK  protocol  errors  associated 
with  the  RA/-1  (Uplink  Display  Message)  and  RA/-4  (Loop-Back) 
uplink  message  labels  noted  in  prior  flight  testing.  Each  RA/-1 
received  both  an  ACK  and  a  NAK  general  response  downlink  for  the 
first  uplink  block  of  a  single  or  multi-block  uplink,  while  the 
RA/-4  received  an  ACK  for  the  first  block  and  both  an  ACK  and  a  NAK 
for  any  additional  blocks.  In  each  case  the  avionics  caused  excess 
use  of  the  available  RF  resources  and  suffered  adversely  (77%)  in 
terms  of  uplink  success  ratios  due  to  the  multiple  uplink 
retransmissions  necessary  to  deliver  each  RA/-l,-4  labeled  message. 
An  ASCII  format  floppy  disk  of  the  audit  data  has  been  forwarded  to 
Rick  Olson  of  the  FAA  to  help  illustrate  the  observed  protocol 
errors.  The  ACK/NAK  problem  is  a  definite  protocol  error  and  could 
in  connection  with  the  normal  timeout  and  retransmission  intervals 
of  the  system  affect  the  uplink  data  link  integrity  for  the  FAA. 


MESSAGES  OBSERVED: 


The  following  uplink  and  downlink  messages  were  observed  during  the 
audited  flights: 


Label 

Sublabel 

U/D 

DEL 

n/a 

U 

•  • 

•  § 

n/a 

U 

RA 

-1 

U 

RA 

-2 

U 

RA 

-3 

U 

RA 

-4 

U 

Cl 

n/a 

U 

HI 

DF 

U 

51 

n/a 

U 

Message  Type 


General  Acknowledgment 

Autotune  to  New  Frequency 

Display  Message 

OOOI  Dump 

Memory  Dump 

Loop-Back  Test 

Display  Message 

DFDAU  Message 

GMT  Update 


A-7 


54 

n/a 

U 

Voice  Go-ahead 

DEL 

n/a 

D 

General  Acknowledgement 

NAK 

n/a 

D 

Non-Acknowledgement 

F3 

n/a 

D 

Dedicated  Transceiver 

QC 

n/a 

D 

ON  Report 

QF 

n/a 

D 

OFF  Report 

Q0 

n/a 

D 

Link  Test 

Q3 

n/a 

D 

Clock  Update  Advisory 

Q5 

n/a 

D 

Unable  to  Deliver  Message 

RB 

-2 

D 

0001  Dump 

RB 

-3 

D 

Memory  Dump 

RB 

-4 

D 

Loop-Back  Test  Response 

5U 

n/a 

D 

Heather  Request 

5Z 

ACK 

D 

Manual  Acknowledgement 

5Z 

TXT 

D 

Free  Text  Message 

51 

n/a 

D 

GMT  Update  Request 

54 

n/a 

D 

Voice  Contact  Request 

AVIONICS  UPLINK  PERFORMANCE: 

The  following  uplink  success  ratios  were  observed  during  the  audit. 
These  ratios  are  a  measure  of  the  reliability  of  an  individual 
message  being  received  by  the  avionics  on  the  first  uplink 
transmission  occurrence.  The  observed  ratios  were  as  follows: 

Solicited  Uplink  Successes:  80%  (16  Successes/20  Attempts) 

Unsolicited  Uplink  Successes:  76%  (36  Successes/47  Attempts) 

Overall  Uplink  Successes:  77%  (52  Successes/67  Attempts) 

As  shown  above,  the  success  ratios  are  below  the  80%  minimum  AQP 
requirement.  The  majority  of  the  failed  uplink  attempts  were  due 
to  the  retransmissions  effected  by  the  unnecessarily  NAK'd 
messages.  If  these  extra  messages  were  not  included  in  the  success 
ratio  calculations  above,  the  overall  uplink  success  ratio  would 
increase  to  96%. 


NOTED  PROBLEMS  AND  ISSUES: 

The  problem  noted  in  previous  testing  with  the  ACK/NAK  protocol, 
sending  both  a  positive  acknowledgement  and  a  non-acknowledgement 
downlink  as  a  response  for  a  single  uplink,  was  again  evident 
during  this  audit. 

•  During  the  1.5  hour  audit,  each  of  the  three  RA/-1  (Free  Text 
Uplink)  messages  sent  to  the  avionics  was  originally  ACK'd  by 
the  avionics  on  the  first  uplink  occurrence  and  then 
improperly  NAK'd  five  seconds  later.  Two  of  these  occasions 
were  multiple  blocked  uplinks.  On  each  of  the  multi-block 
uplinks,  only  the  first  block  received  the  unnecessary  NAK. 


In  each  case,  the  second  block,  while  not  NAK'd  itself,  had  to 
be  resent  due  to  a  five  second  delay:  the  first  block  KAK 
arriving  at  AFEPS  after  the  second  block  had,  because  of  the 
receipt  of  the  first  block  ACK,  already  been  sent.  Upon 
receipt  of  these  erroneous  NAKs,  AFEPS  interpreted  then  to 
mean  that  the  second  block  was  in  error,  which  caused  the 
second  block  to  be  resent  unnecessarily. 

•  Each  of  the  six  nulti-block  RA/-4  (Loopback)  tests  that  was 
initiated  during  the  audit  received  an  ACK  for  the  first  block 
and  both  an  ACK  and  a  NAK  for  any  subsequent  blocks  at  the 
same  five  second  delay  interval.  In  each  case,  the  five 
second  delay  propagated  through  the  message  causing  the  last 
block  of  each  message  to  be  resent  two  to  three  times. 

•  Both  an  RA/-2  (0001  Dump)  and  an  RA-3  (Memory  Dump)  were 

observed  to  operate  properly  without  any  additional  NAKs 
attached.  These  were  single  block  command  response  uplinks 
and  only  tested  once. 

In  summary,  The  RA/-1  messages  received  an  ACK/NAK  for  only  the 
first  block,  the  RA/~4  messages  received  an  ACK/NAK  for  any  blocks 
beyond  the  first.  The  responses  to  the  RA/-1  and  RA/-4  uplinks  by 
the  avionics  are  serious  deviations  from  any  of  the  published  AEEC 
standard  characteristics  and  were  observed  to  cause  multiple 
unnecessary  uplink  retransmissions. 


John  Linsenmeyer, 
QM/QTST 
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